RepLab 2013: Comparative Evaluation of Online Reputation Monitoring Systems and Components

We summarize the goals, organization, and results of the second RepLab competitive evaluation campaign for Online Reputation Management systems (RepLab 2013). RepLab 2013 focuses on the process of monitoring the reputation of companies and individuals, and asks participating systems to annotate different types of information on tweets containing the names of several companies. First, tweets have to be classified as related or unrelated to the entity; relevant tweets have to be classified according to their polarity for reputation (Does the content of the tweet have positive or negative implications for the reputation of the entity?), clustered in coherent topics, and clusters have to be ranked according to their priority (potential reputation problems had to come first). The gold standard consists of more than 140,000 tweets annotated by a group of trained annotators supervised and monitored by reputation experts.